Validation of the patient reported experiences and outcomes of safety in primary care compact Form Brazil

Objective To analyze the psychometric properties of the cross-culturally adapted version of the Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS–PC) Compact Form Brazil. Methods A methodological study was conducted with 281 adult Primary Health Care users. Data collection took place online. Confirmatory Factor Analysis (CFA) was used to evaluate the psychometric properties of the PREOS–PC after the process of cross-cultural adaptation to the Brazilian context. Internal consistency was evaluated through Cronbach’s alpha coefficient (α) and McDonald’s omega coefficient (ω). Results The sample consisted of 73.3% women. The mean age was 36.1 years (SD = 12.2). Of the 23 items of the PREOS–PC that were eligible for CFA, a model with four correlated domains and 16 items presented satisfactory fit indexes. The domains were Practice Activation (PrA) (four items), Patient Activation (PaA) (two items), Experiences of patient safety events (EPaS) (five items) and Outcomes of patient safety (OPaS) (six items). One domain (GPeS) presented one question with a 0 to 10 response scale and two open questions, which cannot be inserted in the CPA due to the nature of the items, but can be included in the application of the scale, being evaluated individually. In this factorial model, five items (EPaS2, EPaS3, EPaS4, EPaS5, EPaS6 and EPaS8) presented factor loadings ≤ 0.30. The α and ω values demonstrated good internal consistency for all domains of the PREOS–PC. Conclusions The Brazilian version of the PREOS–PC Compact Form Brazil composed of four domains (PrA, PA, EPaS and OPaS) and 16 items presented evidence of validation of its psychometric properties and can be used to evaluate the experiences and results of patient safety in Primary Health Care in the Brazilian context.


Background
Patient safety (PS) is conceptualized by the World Health Organization as "the reduction of risk of unnecessary harm associated with healthcare to an acceptable minimum" [1].Patient safety consists of a set of organized actions that influence healthcare cultures, processes, behaviors, technologies and environments, with the aim of reducing adverse events, errors and their impacts on patients, families, professionals and health services [2].
Most of the research, discussions and efforts that have sought to understand the nature and magnitude of adverse events, evaluating the experiences and results of PS, have been focused on hospital contexts, care environments in which activities are more standardized, complex and carry a higher risk of adverse events [3].Furthermore, previous research has focused on the experiences and outcomes in the perception of healthcare providers rather than patients [4][5][6].
Studies that have evaluated these experiences in Primary Health Care (PHC) are limited, especially in relation to the perception of patients [3,7].At this level of care, errors related to the use of medications, misdiagnoses and treatments, problems in communication and in the relationship of the healthcare provider with the patients are frequent, which can contribute to increase the magnitude of adverse events [8].In low-and middle-income countries, the prevalence of PHC safety incidents can vary from 25 to 40% of the care provided, with 80% of these incidents being preventable [9].Estimates indicate that about 20% and 25% of the general population suffers damage in this scenario in developed and developing countries, respectively [9].Therefore, unsafe primary care can cause higher morbidity and mortality and increased costs for health services and countries [10].
The experience reported by patients is crucial to improve PHC services, with the aim of improving patient care and promoting the PS culture.This form of evaluation facilitates user involvement, healthcare planning for the improvement and delivery of safe care, increased communication, and trust in healthcare providers [11,12].Additionally, patients and their families can assist in the identification of failures in PS or in the quality of care [13].
The use of validated and reliable scales to measure and monitor PS in PHC is indispensable.Some methods are immediately available, easy to administer, do not require external involvement and have low cost [14].Among them, the Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC) is a multidimensional instrument developed by specialists and patients that has shown potential for implementation in PHC and use in research.The original instrument, in its final version, has 58 items, with several grouped into five PS domains: a) Practice Activation (PrA), b) Patient Activation (PaA), c) Experiences of patient safety events (EPaS), d) Outcomes of patient safety (OPaS) and e) General perceptions of safety (GPeS) [7].
The PREOSP-PC analyzes the experiences of patients and the results of PS problems in PHC, allowing the differentiation between the different levels of PS between practices and over time in this level of care [7].It presents three versions, one screening form, one compact form and one comprehensive form, which can be used independently to evaluate five PS domains in PHC in a unique way [15].The validation study of the comprehensive version was carried out in England with 6,736 patients.Psychometric methods were applied to evaluate its acceptability, reliability, and structural and construct validity, presenting satisfactory parameters [7].Despite its relevance, this instrument has not yet been cross-culturally adapted and validated for the Brazilian population, which limits its use in the country.
The compact form of the PREOS-PC provides a balance between high psychometric standards and a reduced administrative burden, facilitating its implementation and use in the real practice.Therefore, by validating the scale in PHC care environments in Brazil, it will be possible to identify which experiences are most reported by patients and which PS-related problems need greater attention and, consequently, interventions.The analysis of patients' experiences may allow an analysis of the current state of PS in PHC, providing relevant data for managers, aiming to develop improvements that meet the needs identified and improve the quality of the services.These actions would have a direct impact on patients' lives due to the improvement of PS in PHC.
The aim of this study was to analyze the psychometric properties of the cross-culturally adapted version of the PREOS-PC Compact Form Brazil.

Outline
Methodological study to verify the psychometric properties of the PREOS-PC Compact Form Brazil, translated and cross-culturally adapted from the PREOS-PC [7,16].Methodological studies deal with the development, validation and evaluation of research tools and methods, such as PREOS-PC Compact Form Brazil.

Cross-cultural adaptation
The instrument was cross-culturally adapted according to the recommendations of Oxford University Innovation Limited [17] and the methodological guidelines of Beaton et al. [18,19].It is important to mention that we obtained the necessary permissions from the owner of the original questionnaire (https://innovation.ox.ac.uk/wp-content/uploads/2018/07/PREOS-PC-27-item-version-SAMPLE.pdf) to carry out the translation and cross-cultural adaptation of the instrument to Brazil.
The process of cross-cultural adaptation of the PREOS-PC Compact Form Brazil was carried out in several stages.First, the questionnaire was translated into Brazilian Portuguese (the language of the participants).The instrument was translated into the target language by two independent native Brazilian translators, with experience in this type of study, one of them with experience in scientific health terminology and the other in linguistics and culture of the Brazilian Portuguese language.The translation of the PREOS-PC instrument into Brazilian Portuguese was carried out in May 2019.Secondly, the synthesis of translations was performed by a third independent bilingual translator, with knowledge and expertise in research in the health area, generating an initial version.This stage was carried out in June 2019.Thirdly, in August 2019, this last version was back-translated by two independent professional translators (with British English as their native language, but fluent in Portuguese), who were not aware of the original version of the assessment instrument or the objectives of the study, which resulted in two more versions.
Fourth, all versions of the instrument were evaluated and approved by an expert committee composed of members of the research team who are experts in the field in question.The review by a committee of experts aimed to evaluate and consolidate all translation and backtranslation versions of the instrument, with the aim of developing a single version of the instrument, in the target language, which would be used in the subsequent steps [18].
Finally, a pre-test (pilot) was performed using the final version of the instrument with 30 participants, aiming to evaluate the ability of individuals to interpret and understand the items of the instrument and to provide measures of internal consistency and content validity.This stage was carried out between August 1st and October 30th, 2020.Participants provided consent online, by reading the informed consent form and subsequently accepting the term.After these steps, the participant had access to the questionnaire.The participants in this stage were individuals from two PHC units with a mean age of 42.0 years.The majority (80.0%) were female.
The results of the pre-test showed good understanding of the items by the participants.The McDonald's omega coefficient (ω) [20] was used to evaluate the internal consistency of the scale, with acceptable values above 0.6.The ω found were 0.689 for the practice activation domain, 0.749 for the patient activation domain and 0.850 for the PS results, demonstrating good internal consistency of the translated and cross-culturally adapted version.
Content validity was also performed by a committee of eight expert judges who evaluated the semantic (verifying whether the words have the same meaning in the original and translated versions), idiomatic (observing expressions difficult to translate that are not suitable for the place of study, to then elaborate an equivalent expression in the target language version), conceptual (verifying whether the words present a different conceptual meaning between cultures) and cultural (trying to find terms that correspond to those described in the instrument and that express situations or activities of daily life in the culture, so that the situations described in the instrument are in accordance with those experienced in the context of the local culture) equivalences of the final instrument [18,19].This stage was carried out between November 1st and December 29th, 2020.Each judge evaluated the items through a Likert-type response scale from 1 to 4 (1 = not relevant or not representative, 2 = need for major revision to be representative, 3 = needs minor revision to be representative, 4 = relevant or representative).The mean Content Validity Index (CVI) of 0.96 was calculated, being considered satisfactory (CVI�0.80)[21].
The participants of the committee of judges were selected from searches on the Lattes Platform of the National Council for Scientific and Technological Development (CNPq) for health professionals with command of Brazilian Portuguese and English; with a master's or doctorate degree; with experience or work in the area of quality and patient safety or in primary health care; with scientific production and experience in research involving the application of instruments or recognized knowledge in cross-cultural adaptation of measuring instruments, that is, with methodological knowledge in the preparation and/or adaptation of instruments.To avoid regional biases, the diversity of location and place of birth of the judges was considered, which is notably relevant in Brazil, given the large geographic dimension and its regional disparities.An invitation to participate in the committee was sent via email to each specialist.Thus, the committee was made up of eight experts.All judges with specialization in patient safety have already published works on adapting and validating instruments, and only two were not part of a working group related to patient safety.
Of the eight judges, half were nurses, and 7 had a doctorate as their highest degree, in terms of specialty, 4 were specialists in Patient Safety, 3 in measurement/psychometrics and 1 in primary health care.The average time working in the respective specialty areas was 10.25 years.As for the region of Brazil, 4 were from the Central-West, 2 were from the South, 1 was from the Southeast and 1 from the Northeast.
The flowchart with the stages of cross-cultural adaptation is shown in Fig 1.
The final version of the instrument is available in S1 File.

Participants
The population was composed of users of the Brazilian public health system (Sistema Único de Saúde-SUS) attended in PHC.Adults aged 18 years or older, of both sexes, native speakers of the Brazilian Portuguese language, registered in one of the 40 units of the PHC network of Goia ˆnia, who had used primary health services at least once in the previous 12 months, regardless of the reason, were included.Individuals without internet connection by mobile phone and/or who did not use the free messaging application WhatsApp were excluded, since the instrument was self-administered via google forms, through a link to access the questionnaire sent through the application.

Sample size
The minimum sample size was estimated according to the need for five to ten respondents for each item of the instrument [25].Considering that the PREOS-PC Compact Form is composed of 27 items, a minimum sample size of 270 individuals was required.

Data collection
Data collection took place through an online access platform.The participants answered two self-administered instruments.The first included the collection of sociodemographic data on health and use of PHC health services in the previous 12 months.This instrument was adapted from questions used and validated in another study [26].The second included the application of the cross-culturally adapted PREOS-PC Compact Form Brazil.The selection of participants was performed using the simple random sampling technique, from the list of medical records of the PHC units, where the necessary contact information was obtained, such as the mobile phone number.Due to the pandemic of the disease caused by the SARS-CoV-2 virus (COVID-19), users were contacted by phone and invited to participate in the study.The data collection instruments were also transferred to the Google forms platform and the survey was made available to receive the online responses of the participants through WhatsApp.By accessing the link, the participant was able to view the information about the study and, to proceed, they had to agree to participate in the research.If calls are not answered by the participant, new calls were made to people treated at the same health unit

Measurements
Sociodemographic variables, health conditions and use of health services.The sociodemographic data collected included age (in years), gender (male or female), self-declared race/ skin color (white, mixed race, black or other-Asian or indigenous), schooling (up to complete elementary school, incomplete/complete high school or incomplete/complete higher education), family income (minimum wages in reais-R$) (< 1, 1-2 or > 2).Data were also collected on comorbidities (no or yes), use of medications (no or yes), considering the use of any medications as yes [27], and self-assessment health (very bad/bad or good/very good).Data were collected on the use of health services, the number of times they had used the unit's services in the previous 12 months (1, 2, 3 or �4) and the number of consultations (with a doctor and/or nurse) in the unit in the previous 12 months (1, 2-4 or �4).
PREOS-PC Compact Br.The PREOS-PC Compact Form Brazil was applied to obtain the necessary data for Confirmatory Factor Analysis (CFA).
The instrument presents eight questions and 27 items of different types, including 12 Likert-type items, nine multiple-choice items in which the individual needs to mark one or more items from a checklist of experiences with safety problems, three dichotomous items (yes and no) and two open questions.Question 1 presents four items (PrA1 to PrA4) with Likerttype responses (always, often, sometimes, rarely and never), with the option 'does not apply'.These items are grouped in the PrA domain.Question 2 presents two items (PaA1 and PaA2) with Likert-type responses (always, often, sometimes, rarely, never), with the option 'does not apply'.These items are grouped in the PaA domain.Question 3 presents nine multiple-choice items (EPaS 1-9).It should be noted that this question presented an item referring to "None of the above options", which was excluded from the CFA.Question 4 presents three items (EPaS 10-12), with dichotomous answers (yes and no).The items of question 3 and 4 are grouped in the EPaS domain.Question 5 presents six items (OPaS 1-6) with Likert-type responses (no, yes, some; yes, many; yes, extreme levels), as well as 'I don't know'.These items are grouped in the OPaS domain.Finally, question 6, in the GPeS domain, is of the closed type (GPeS1) on a scale from 0 to 10 and questions 7 (GPeS2) and 8 (GPeS3) are open.Questions GPeS1, GPeS2 and GPeS3 were not included in the CFA (Table 1).In the original model, 24 items (excluding items EPaS 9, GPeS1, GPeS2 and GPeS3 due to their response natures) were part of the factorial structure of the scale and were included in the CFA.
The Anderson-Darling test was initially performed to verify the normality of the quantitative variables.Next, a descriptive analysis of the variables was performed.Quantitative variables were presented as mean and standard deviation.For the PREOS-PC Compact Form Brazil, the median, 25 th percentile (P25), 75 th percentile (P75), minimum and maximum values were also presented.Qualitative variables were presented as absolute and relative frequency in percentage.
Construct validity was determined using CFA.This statistical technique aims to evaluate the relationship between latent variables (also called domains) and their relations with a preestablished theoretical covariance structure [29].The Mardia normality test was performed to evaluate the multivariate normality of the PREOS-PC Compact Form Brazil [30].Due to the absence of multivariate normality (p-value < 0.001), a CFA with likelihood estimation with robust variance was performed [31].Two different factor structures of the PREOS-PC Compact Form Brazil were examined.The initial model of the scale was adjusted through firstorder CFA, with four independent domains (PrA, PaA, EPaS, OPaS) and correlated with all 23 eligible items.Next, a final model was established by means of first-order CFA, with four independent and correlated domains, totaling 17 items.In the final model, only items with a factor loading > 0.30 in the initial model remained.
The following indices were used to verify the quality of fit of the models: (i) chi-square fit statistics (χ 2 )-non-significant p-values (�0.05) indicate good quality of fit [32]; (ii) Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) with acceptable values equal to or greater than 0.95 [33] and (iii) Root Mean Square Error Of Approximation (RMSEA)values < 0.06 indicate good fit, 0.06-0.07acceptable fit, 0.08-0.10limited fit, and > 0.10 inadequate fit [33].
After the estimates, the initial and final models were compared by the Satorra-Bentler Scaled Chi-Square test, which measures the difference between χ 2 and the degrees of freedom (df) between the original and alternative models.The model chosen, if there was a statistical difference, presented the lowest chi-square rate, in addition to fit indices within the cutoff points [34].
The internal consistency of the instrument was evaluated through Cronbach's alpha (α) and McDonald's omega (ω), followed by the respective 95% confidence intervals (95%CI).These coefficients can range from zero to one, and the closer to one, the greater the internal consistency of the instrument.Values of α > 0.70 indicate good internal consistency [35], while values of ω > 0.6 indicate good internal consistency [20].

EPaS10
Communication problem between you and healthcare professionals (for example, not receiving the information you need to take care of your health).

EPaS11
Communication problem between professionals on the healthcare team (for example, not sharing important information about your health or healthcare between professionals in the unit).

EPaS12
Communication problem between health professionals on the unit and health professionals outside the unit (for example, not sharing important information about your health or health care with professionals who work in the specialty outpatient clinic or hospital).The Satorra-Bentler Scaled Chi-Square test for differences between the initial and final models suggested a better fit of the final model to the factor structure.The p-value of the test was 0.0059, which demonstrates that the model with 17 items was significantly higher than the complete model with 23 items.

Discussion
The present study provides evidence on the psychometric validity of the PREOS-PC Compact Form Brazil with a sample of PHC patients.This instrument has been used in other countries, but, to date, its validation had not been performed and its applicability remains limited in the Brazilian context.The results demonstrate the validity of the PREOS-PC Compact Form Brazil, with a factorial structure composed of 17 items and four correlated domains (PrA, PaA, EPaS and OPaS).The instrument presented good internal consistency.Therefore, it represents evidence of construct validity to measure PS perceptions, experiences and results from the perspective of PHC users.
The initial model (with 23 items and four correlated factors) presented low factorial validity indexes.However, after the exclusion of six items, the model proved to be adequate, with good fit indexes.The excluded items were EPaS2, EPaS3, EPaS4, EPaS5, EPaS6 and EPaS7, all dichotomous and belonging to the EPaS domain.They relate to a single question that covers the problems that patients may have faced during PHC care.The inclusion of dichotomous items in the CFA model may result in biased parameter estimates with inaccurate interpretation limitations [36].Therefore, the absence of contribution to the domains of these items may have been caused by the very nature of the items and, not necessarily, because they did not contribute to the formation of the construct.Another point is that some excluded items relate to health practices that are uncommon in the context of PHC in Brazil, particularly the administration of medications in the health unit, treatments, blood tests, laboratory tests and diagnostic tests, which are predominantly performed in the secondary (emergency and urgency units) and tertiary (for example: hospitals) networks [37].This can decrease the probability of the patient's knowledge about aspects related to the problems faced during their PHC care.All the excluded items, in fact, presented low prevalence in the sample, which may have contributed to their non-fit in the domains.In addition, the collection was carried out in the context of the COVID-19 pandemic in a scenario of reduced supply of health services [38], which may have contributed to the low prevalence of these items.The final CFA model showed four correlated domains: "PrA", "PaA", "EPaS" and "OPaS".The first contains items that evaluate what the health unit does to create a safe environment and ensure PS.The second presents items that seek to understand how patients are proactive to ensure that the healthcare is safe.The third has items that assess the prevalence of safety errors related to healthcare.The fourth presents items that assess the presence of health damage experienced by users.A domain that is not adjusted to the CFA due to the nature of the items, is called "General perceptions of patient safety", which aims to evaluate how safe patients perceive their health unit to be and should be analyzed in isolation [26].
Internal consistency was assessed using Cronbach's α and McDonald's ω.The values were adequate, higher than recommended (α > 0.70 and ω > 0.60) [20,35].McDonald's ω is a measure of solid internal consistency, but is little used in scale validation studies [39].The PREOS-PC is relatively recent.No studies were found that analyzed the internal consistency of the instrument, along with the psychometric properties of the compact version.To date, the reliability of the instrument has only been tested for the full version, using Cronbach's α, but not McDonald's ω. [7].The results of the full version study showed high internal consistency for the domains (Cronbach's α: PrA = 0.89; PaA = 0.80, EPaS = 0.75 and OPaS = 0.96) [7].Although with differences in the items, the internal consistencies were similar in the present study (Cronbach's α: PrA = 0.90; PaA = 0.82, EPaS = 0.74 and OPaS = 0.94).In the original study, the EPaS domain presented the lowest internal consistency value, which is like that found in this study, which may explain the removal of some of the items from the final model.
It is important to note that the PREOS-PC Compact Form Brazil is a reduced version and, to date, there are no studies that can accurately compare this version, since the original extended instrument has 58 items [7], which prevents a more accurate comparison with the literature.The validity of the structural construct of the General Perceptions of Patient Safety (GPeS) domain was not confirmed by factor analysis, since one item was composed of a scale from 0 to 10 and two were open questions.Therefore, the compact version does not allow a factorial evaluation of the instrument, since it does not include this domain.It is important to note that, in the full version, the "General perceptions of patient safety" domain presents a single scale question and two open questions [7].
The present study has some limitations.Data collection was performed in a single Brazilian city, which limits the ability to generalize the results.The experiences and reports of the patients in these PHC units may not be the same as the practice in the rest of the country.The study sample was composed of 73.3% women, which does not guarantee that the results found are not distorted by the lack of sampling balance between the sexes, and may not be generalized to men [40].Additionally, the data was collected online, due to the COVID-19 pandemic.These individuals may have characteristics that differ from people who do not have access to the internet, which also limits generalization.In addition, the validation study of the full version was performed through the application of the physical instrument.Some calls went unanswered, which may have led to selection bias.All measures were self-reported, susceptible to memory and response biases.The inclusion of dichotomous items in the CFA is another limitation, which reduces the contribution of these items in the latent domain.
However, the study has strengths.First, the relatively large sample size allowed the inclusion of enough patients to conduct CFA.Second, this psychometric validation study is unprecedented in Brazil, being a starting point for future investigations that may be carried out in other Brazilian regions, as well as the implementation of the instrument in the PHC practice of the country.The relevance of this study is the possibility of improving the quality of care in PHC in Brazil, adapting methodologies to PS perceptions, experiences and results, through the evaluation of the patients themselves.This study can also help in the implementation of a process to improve the PHC care environment, as well as consolidate knowledge about the theme in the literature.However, further studies are still needed to evaluate the psychometric properties of the PREOS-PC Compact Form Brazil, especially in relation to some of the less prevalent items.Accordingly, future studies with more diverse samples should be carried out to have a clear understanding of users' perceptions about PS in PHC, so that other comparisons and generalizations can be made.It is important to emphasize that, especially, conducting studies in other Brazilian regions, in a self-administered and face-to-face manner, could contribute to the improvement of the instrument and to the identification of new tests of validity and reliability for the PREOS-PC Compact Form Brazil in different contexts.

Conclusions
The Brazilian version of the PREOS-PC Compact Form Brazil presented satisfactory validity and psychometric properties for 16 items and four domains (PrA, PaA, EPaS and OPaS).The items EPaS2, EPaS3, EPaS4, EPaS5, EPaS6 and EPaS7 presented low factor loadings, which indicates that, in the Brazilian scenario, they do not contribute to the latent variables of the domains.However, these items should be incorporated into future psychometric studies for further assessment.Furthermore, the isolated analysis of the GPeS domain items is not suitable for CFA but should be considered when assessing PS in PHC.The instrument showed good internal consistency.

Table 1 . Coding of items and response options of the PREOS-PC Compact Form Brazil.
suggestion to doctors, nurses, or other health professionals at the Basic Health Unit or Health Care Unit Family Health when you thought something could be done to improve the service provided.
EPaS5Blood tests and/or other laboratory tests requested or carried out at your Basic Health Unit or Family Health Unit.Checklist (yes).Otherwise, no.EPaS6Requesting or carrying out other diagnostic and follow-up exams at your Basic Health Unit or Family Health Unit, such as electrocardiogram (ECG), tomography, and X-ray, among others (except blood tests and/or other laboratory tests).

Final model of confirmatory factor analysis of the construct under study of the compact PREOS-PC Compact Form Brazil.
https://doi.org/10.1371/journal.pone.0305414.g002